Perceptron Linear Algebra

Introduction

R Packages

Before we begin, we can go ahead and import the required R packages. If they are not currently installed, that can be done first. The packages used in this project are: - data.table - ggplot - plotly

# Install if needed
if (!require(data.table)) install.packages('data.table')
if (!require(ggplot2)) install.packages('ggplot2')
if (!require(ggplot2)) install.packages('ggplot2')

library(data.table)
library(ggplot2)
library(plotly)

Math Fundamentals of the Perceptron Algorithm

In order to understand the Perceptron algorithm, a basic knowledge of the following mathematical principles must first be understood: - Vectors (specifically, how to calculate the direction and norm) - The Dot Product

I am going to assume that majority of people reading this article have prior knowledge of each of these fundamentals. If you do not, a great resource can be found here. I will also give a brief refresher to provide some context.

Vectors

The Dot Product

Linear Separable Data

The definition of linearly separable data is pretty self-explanatory. At a high level, for data to be linearly separable it simply means that there exists some plane that can literally “separate” the data. For example, consider the following plot:

It is pretty obvious that you can draw some arbitrary line that separates the two different colored data points:

Similarly, consider the following three-dimensional data:

In this instance, a line would be insufficient when attempting to seperate the two data classes. Instead, a two-dimensional plane can be used:

Of course, this gets a little harder to visualize - or even explain - as dimensionality continues to increase. As a result, a more generalized term to describe this linear separation is used: a hyperplane.

Hyperplanes

According to the Wikipedia definition: “In geometry, a hyperplane is a subspace whose dimension is one less than that of its ambient space.” This makes sense. Looking at our first example, the deminsionality of the data is two, and the line drawn to separate the data is one-dimensional. In the second example, the data is three-dimensional and the separation is two.

Perceptron Algorithm

Generate Data

Weights

Iterate

Multi-Layer Perceptron